Content Moderation

Research

EFF

From the EFF's "Platform Censorship: Lessons From the Copyright Wars", Corynne McSherry makes some points:

• Mistakes will be made—lots of them. There's genuine errors and abuse by rightsholders and takedown trolls
• Robots fail
• "Platforms must invest in transparency and robust, rapid, appeals processes". ie "every proposal and process for takedown should include a corollary plan for restoration"
• "Abuse should lead to real consequences"
• Speech regulators will never be satisfied with voluntary efforts:

Community Moderating – Philip Sheldrake

https://blog.ethereum.world/community-moderation-myths-goals/

• Myth 1: moderation is censorship: "the corollary of freedom of speech is freedom of attention"
• Myth 2: moderation is unnecessary: "Large scale social networks without appropriate moderating actions are designed to host extremists, or attract extremists"
• Myth 3: moderation is done by moderators: "If a community member is active, she is moderating. In other words, she is helping to maintain and evolve the social norms governing participation."
• Myth 4: moderation is simple enough: or rather moderation design "is as much art as science. Complexity refers to the phenomena whereby a system can exhibit characteristics that can't be traced to one or two individual participants" Fopr e.g.:

   - A team of central moderators that just can't keep up with the volume of interactions requiring their attention
   - The value of engaging in moderating processes is considered insufficient
   - Moderating processes are perceived as unfair
   - Those doing the moderating cannot relate to the context in question
   - Moderating processes are too binary (e.g. expulsion is the only punishment available).
  

  Two Concepts of Liberty and Infinite Permutations of Moderating

  "We need to figure out together how best to accommodate multiple contexts and naturally varying expectations; how much variation can one 'world' satisfy, and when might a new 'world' be necessitated?""

  Requisite Variety & Complexity (ie stop simplifying)

  "The Law of Requisite Variety asserts that a system's control mechanism (i.e. the governing, specifically the moderating in the context here) must be capable of exhibiting more states than the system itself. Failure to engineer for this sets the system up to fail." Philip Sheldrake ref "Boisot and McKelvey updated this law to the ‘Law of Requisite Complexity’, that holds that, in order to be efficaciously adaptive, the internal complexity of a system must match the external complexity it confronts. A further practical application of this law is the view that information systems (IS) alignment is a continuous coevolutionary process that reconciles top-down ‘rational designs’ and bottom-up ‘emergent processes’ of consciously and coherently interrelating all components of the Business/IS relationships in order to contribute to an organization’s performance over time" https://en.wikipedia.org/wiki/Variety_(cybernetics)#Law_of_requisite_variety

Issues & reflections

• GDPR, Doxxing & the Streisand Effect – e.g. JK Rowling's & and Barbara Streisand's house (https://en.wikipedia.org/wiki/Streisand_effect). ie legal rulings on privacy and/or harassment and the impact of exerting those rights on public women. Wikipedia is regularly battling with an attempt to disclose JK Rowling's address.
• Append-only logs, ie DHTs, cannot handle historic data/moderation breaches/failures.
• What are the defaults? Control and configuration is used by a majority; most use the defaults. So a moderation solution can be sold on the basis it allows a wide range of actors to make informed decisions - but for most users they'll use the default setting, which will sit someonewhere on the spam <-> censorship spectrum.
• No system is going to be failsafe. Other than ISP-based censorship/controls (and that can be bypassed with VPNs and TOR). In other words all moderation needs to start from the sober acknowledgement that anyone wishing to share or access illegal material on the Internet will be able to, and that this can be linked to and referenced (and that perfectly legal links can later become illegal as what's linked to changes / and vice versa).
• Whose job is policing, and which country are they from? This leads to a tension between a platform in the role as police, versus as enabler. Some platforms might argue that system level control prevents users from acknolwedging their own legal liability online - in what they access, share and do – as distinct from 'what they can get away with'. ie just because Twitter allows someone to racially harass a public figure, doesn't make the behaviour exempt from laws on harassment, hate speech or discrimination (nor does it exempt Twitter for supporting such activity). In otherwords, "just because you can pretend to be a dog, doesn't mean you're not a prosecutable citizen online". Inevitably the defaults and systems that are used to handle moderation will bias towards the legal system of the operator; YouTube is built around the US idea of fair use, not Germany or Japan's.
• Anonymity is a right to privacy, not a right to break the law

(Proposed) systems

Clean up the Internet

https://www.cleanuptheinternet.org.uk/ A simple proposal to limit harassment online with three steps:

• allow everyone on a social network to verify they are a real, legal person and have this displayed.
• allow everyone on a social network to control their interactions with unverifieid people. Clean up the Internet has been central in shifting the debate around anonymity in the UK from the polar view of "defenders of the platforms’ current “anything goes” stance, and those seeking an outright ban" (ref.

OcapPub: Object Capabilities & networks of consent.

https://gitlab.com/spritely/ocappub "“networks of consent”: explicit and intentional connections between different users and entities on the network. The idea of “networks of consent” is then implemented on top of a security paradigm called “object capabilities”, which as we will see can be neatly mapped on top of the actor model, on which ActivityPub is based."

• An example is given: known contacts can message each other, unknown contacts must pay a refundable postage stamp fee, refundable if the receiving party appreciates/approves the message, thus reducing automatic bots and bad actors. A similar system could be used to add a moderation fee on content that is refunded if media is approved but kept if its rejected.
• Object Capabilities underpin such functionality.. which are different from Access Control Lists in that they assign specific functions (e.g. "share a video in the 'great video list'") to specific users, rather than general roles which have a group of individual functions (editor, moderator, etc). In principle it allows much more granular control - e.g. trust person X ('Great Auntie QAnon Addict') to share their cat memes but don't want to see their views on vaccines or health information. This overlaps with 'Trust areas' described in TrustNet - that we trust people in some areas but not others.

TrustNet: Trust-based Moderation as used on Secure Scuttlebut

https://www.cblgh.org/dl/trustnet-cblgh.pdf - Alexander Cobleigh, University of Lund, dept of Automatic Control - thesis explores how to handle moderation in a decentralised context.

• stars from position that typical choices of 'leave it to individuals' (too big a burden) and 'appoint moderator' (reinstates centralised control) aren't enough.
• instead proposes 'what if participants could automatically block the malicious peer if they discover the peer has been blocked by someone they trust'? Ie each member of the network follows a moderator (or more) of their choice - that can be a central moderator but they can chose to follow someone else.
• uses the Appleseed model to rank network peers by their trustworthiness.

Appleseed - network trust algorythm

"Appleseed [Ziegler and Lausen, 2005] is a trust propagation algorithm and trust metric for local group trust computation. Basically, Appleseed makes it possible to take a group of nodes—which have various trust relations to each other—look at the group from the perspective of a single node, and rank each of the other nodes according to how trusted they are from the perspective of the single node." (ref- the TrustNet essay). Paper: https://www.researchgate.net/publication/2935185_Spreading_Activation_Models_for_Trust_Propagation

Peer Review

One of the oldest and most trusted models for establishing trust in published work. Some proposed innovation around it:

• https://decentralized.science/ "Providing a public repository of Open Peer Reviews and a reviewers’ reputation network"
• This links with https://quartz.to/ - a proposed platform coop around academic publishing.

Watch list and Deny list subscriptions

Many decentralised systems, such as [[Holochain]] offer a list of public keys or IP addresses or user accounts which should be avoided, and whose activities shouldn't be seeded/shared. Users by default follow the deny list of the central administrator, but can set their own overrides, or even subscribe to another user's watch/deny lists.